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ABSTRACT 

A way to assess the large volume of student data 
generated in an introductory graduate-level course in qualitative 
research in education is described. Primary attention is given to 
assessing the technical aspects of field data collection, and only 
secondary attention is given to the cons iderat i on of the substantive 
aspects of the field data. In the research course at Washington State 
University, students are taught basic research knowledge and skills 
as they are guided through an independent and original qualitative 
research study from initiation to completion. The first part of the 
course is concerned with problem conception, proposal development, 
and field data collection. The second section concentrates on the 
analysis of field data, and the final section is concerned with 
reporting the study as a formal technical report with an oral 
presentation. Throughout the data collection phase, students see 
examples of research studies and present their own work in progress 
for assessment. They display their field data after the first 5 weeks 
of the course for evaluation by other students and the teacher. The 
display of the data makes the student accountable to others in the 
class, and this accountability raises student production of quantity, 
quality, and organization of field data to a higher level than would 
otherwise be the case. (Contains 4 figures and 3 references.) 
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ASSESSING STUDENT FIELD DATA IN TEACHING QUALITATIVE RESEARCH 

Introduction 

An introductory course in qualitative research for beginning qualitative 
researchers in education can have at least two different orientations. One 
orientation is concerned with teaching individual research skills associated with 
qualitative research. In this orientation, general and specific skills and related 
knowledge associated with the qualitative research enterprise are taught relatively 
independently of one another. Categories of these skills include proposal writing, 
field entry and data collection, data analysis, and report writing and presentation. A 
second orientation is concerned with teaching students how to conduct and report a 
qualitative research study. In this orientation, students are guided through a 
complete, yet limited, qualitative research study. In the process of conducting and 
reporting the study, students are taught the research, analytic, and writing skills 
necessary to initiate, conduct, and complete an individual qualitative research effort. 
During the data collection phase of the studies, special attention is placed on 
teaching interview processes, participant observation, document collection, 
photographic techniques, and other traditional field data collection methods. 

In this paper, it is the second orientation to teaching an introductory course in 
qualitative research for graduate students in education that is the point of reference. 
Because graduate students in education typically have not had much in-depth 
training in physical, biological, or social sciences, they have little experience with 
and knowledge of how to conduct and report educational research. They are 
research novices in the true sense of the word. Hence, very basic research 
knowledge and skills cannot be taken for granted and must be explicitly taught. 

Such basic research knowledge and skills concerned with the appropriate data 
collection materials and equipment and the organization and management of 
extensive field data must be explicitly taught in an introductory qualitative research 
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course specifically designed for graduate students in education. Equally important, if 
net more important, are the knowledge and skills associated with collecting rich, in- 
depth field data. 

With respect to knowledge and skills regarding data collection materials and 
equipment, field data entry and organization, and the substance of the field data 
itself, two important categories of related problems arise for the teacher of an 
introductory graduate level course in qualitative research in education. One of 
these categories of problems is concerned with instruction; the other is concerned 
with assessment. The instructional problem is concerned with how to instruct 
students in the knowledge of and skills concerned with the technical aspects of data 
collection and management as well as with the substantive aspects of the field data 
itself. The second problem is concerned with how to assess student knowledge of 
and skills in these same areas. Although both problems are of critical importance to 
teachers of qualitative research, the latter is weighted by the sheer volume of data 
and the necessarily rapid turnaround time associated with assessing individual field 
data in a class of ten-to-twenty graduate students. 

Consistent with assessment problems, the purpose of this paper is to describe 
an efficient yet effective way in which a large volume of student field data in an 
introductory graduate level course in qualitative research in education may be 
assessed and simultaneously and/or subsequently evaluated. The focus of this 
paper is primarily concerned with assessment practices; related instructional 
methods are also described but not in as much detail. Within the area of 
assessment, primary attention is given to assessing the technical aspects of field data 
collection; this is to say, organization and quantity of the data. Only secondary 
attention is given to the consideration of the substantive aspects of field data. 

Following this introduction and consistent with the purposes of this paper, 
the paper is divided into three substantive sections and one theoretical section. The 
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first substantive section provides an overview of the course and the students 
typically enrolled in the course. The second section is concerned with instruction; 
the emphasis in this section is on the specific instruction provided regarding data 
collection materials and equipment, field data organization, and field data quality. 
The third substantive section is concerned with assessment. Here the emphasis is 
on efficient and effective assessment practices. And, the fourth section of the paper 
examines assessment from a theoretical perspective. Following the theoretical 
section, the paper ends with a brief conclusion. The final section contains a list of 
the references dted in the paper. 

The Course and Students 

The Course 

The course, Introduction to Qualitative Research in Education, is offered by 
the Department of Educational Leadership & Counseling Psychology in the College 
of Education at Washington State University. It is a three semester-credit course 
and has been offered regularly since the late 1970s. The course is the first in a 
sequence of courses concerned with qualitative research in education. The second 
course in the sequence is entitled Advanced Qualitative Research in Education ; the 
third course in the series carries the title of Special Topics in Qualitative Research in 
Education. The introductory course, about which this paper is concerned, is offered 
twice each year, whereas the subsequent two courses are only offered once each year. 
During the regular academic year, the introductory course is offered fall semester as 
a sixteen-week course which meets once per week in the later afternoons. The 
introductory course is also offered during the summer session as a six-week course 
which meets daily during the early evening. Although over the years a number of 
different texts have served as the foundational reading for the introductory course, 
Robert C. Bogdan's and Sari Knopp Biklen's text (1992), entitled Qualitative 







Research for Education: An Introduction to Theory and Methods, has served as the 
primary text for the course for the past five years. 

The Students 

Typical graduate students in the introductory course are enrolled in a variety 
of degree programs. Only about one-half of them are actually enrolled in degree 
programs in the Department of Educational Leadership & Counseling Psychology. 
But, most of the students are enrolled in degree programs in the College of 
Education. However, the course also attracts a numbers of graduate student beyond 
the College of Education. About two-thirds of the students are enrolled in doctoral 
programs, while the remaining third are enrolled in masters programs. Typical of 
graduate students in education, they have had little or no direct experience with 
empirical research in general or educational research in particular other than some 
exposure to reading about educational research, reading a few reports of educational 
research studies, and perhaps having taken an introductory course in statistics. 

Since the establishment of the course in 1979, enrollment has steadily 
increased to where, as noted above, the course is currently offered twice each year. 
Orientation and Goal of the Course 

As described above, the course is oriented toward guiding students through 
an independent and original qualitative research study from initiation to 
completion. As students are guided though the different phases of a qualitative 
research study, instruction is provided in particular areas of knowledge and specific 
skills associated with qualitative research including proposal and problem 
development, data collection methods, data analysis, report writing, and oral 
presentation. The assumption which stands behind the course orientation is that 
for students to become competent, qualitative researchers, they must each first and 
foremost formulate, conduct, and report a qualitative research study rather than 
learning the variety of skills associated with qualitative research relatively 



disengaged from the actual process of conducting a study. Hence, consistent with 
the orientation of the course, the primary goal of the course is for each student to 
produce an individual and original qualitative research study. This includes 
formulating, conducting , reporting, and presenting an original qualitative research 
study. 

Divisions of the Course 

To achieve the primary goal of the course, the course is divided into three 
related and somewhat overlapping sections. The first section is concerned with 
problem conception, proposal development, and field data collection. The second is 
concerned with analyses of field data. The third and final section is concerned with 
the reporting of a qualitative research study as a formal technical report combined 
with an oral presentation. In terms of the sixteen week (plus one vacation week) 
calendar of the course, about one-third of the time is devoted to each section of the 
course. During the sixteenth and final week of the course, students are required to 
sit for a final course examination. 

It is during the first section of the course that instruction is provided 
regarding field data collection materials and equipment, field data organization and 
management, and the substantive character of field data. It is also during the initial 
section of the course that assessments are conducted first regarding data collection 
materials and equipment, secondly regarding field data organization and quantity, 
and thirdly, and to a lesser extent, regarding field data quality. 

Instruction 

Field Data Materials and Equipment 

It is during the initial class meeting that instruction regarding appropriate 
field data collection materials and equipment is provided. Students in the course 
are provided with a list of the field data collection materials and equipment each 
must have for the course and are instructed to secure the required materials and 
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equipment on the list. A copy of the list is presented in Figure 1. The list includes 
important materials and equipment such as paper, pens, folder, stamps and stamp 
pad, tape recorder with blank tapes, camera and film, and an album for photographs. 
Most of the items on the list are required; however, some are optional. Along with 
the presentation of the list, student are shown physical, individual examples of the 
materials and equipment on the list. Additionally, students are shown and are 
instructed to individually inspect real examples of several complete sets of 
appropriately catalogued and bound and/or filed field data. Following the display of 
individual items on the lists and examples of field data, the student are instructed to 
bring their own required materials and equipment to the second meeting of the 
course for an individual display. 

It is important in this paper to mention the quality and character of the 
instruction given regarding field collection data materials and equipment. On the 
one hand, instruction is explicit and directive. Each student must have each item 
on the list. On the other hand, instruction implicitly attempts to convey a sense of 
flexibility. Students sometimes claim they will be unable to secure particular items 
within the week, often for good reasons. The teacher's response here is, "Just try to 
get the items by next week." 

Studer.t responses to this unambiguous and directive instruction are varied. 
Some claim they will be unable to purchase the required materials during the week 
because they have other obligations. Others claim they have the appropriate 
equipment, but that they have loaned it out and will likely not be able to retrieve it 
in a week's time. It is interesting to note that seldom do students claim that 
bringing their materials and equipment for display is an unnecessary and degrading 
activity. In fact, most students willingly and with mild enthusiasm follow the 
instructions. 
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Figure 1: Field Data Materials and Equipment Assessment Form 

Introduction to Qualitatiue Research in Education Field Data Materials and 
Donald B. Reed Equipment Rssessment 

Name: Date: 

(Print last, first, middle initial) 

Indicate whether you haue secured the following data collection materials 
and/or equipment by circling VES or NO for each Item. 



Item Secured 



1 . 


Field Notebook Couer with Metal Fastener for 
8 1/2" Kir Paper 


VES 


NO 


2. 


Field Note Recording Paper, 8 1/2" K 11" 


VES 


NO 


3. 


Notebook Diuiders, four 


VES 


NO 


4. 


Pen, Black or Dark Blue 


VES 


NO 


5. 


Pen, Red 


VES 


NO 


6. 


Pen, Green (optional) 


VES 


NO 


7. 


Manila Enuelope, Large, for Field Documents 


VES 


NO 


8. 


Date Stamp (optional) 


VES 


NO 


9. 


Stamp Pad (optional) 


VES 


NO 


18. 


Tape Recorder 


VES 


NO 


11 . 


Tape Recording Tapes, 60 Minute 


VES 


NO 


12. 


Camera 


VES 


NO 


13. 


Film for Camera 


VES 


NO 


14. 


Album for Photographs (optional) 


VES 


NO 


15. 


Brief Case, Top Loading (optional) 


VES 


NO 


If certain materials and/or equipment haue not been secured, indicate 1 
when each will be secured. 


below 
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Field Data Substance and Quality 

Instruction regarding the substance and quality of field data begins with the 
second meeting of the class. During the six-week data collection phase, instruction 
is provided regarding traditional field data collection processes and associated data 
gathering techniques. Techniques included various types of interviewing, 
participant observation, document collection, photographing, and conducting 
limited surveys. In class meetings during the data collection phase, students are 
instructed to give weekly, short oral reports regarding their studies and field data 
collection. During these reports, students are asked to read exerpts of their field ata. 
The teacher critiques each exerpt in terms of richness and level of inference. Also, 
during the data collection phase, the teacher reads exerpts from his own research 
data to provide examples of quality and richness of data and low level of data 
inference. Additionally, during this phase, student read about four reports of 
qualitative studies. These reports are discussed in class, and the teacher asks 
students to identify examples of rich data in the studies. Hence, it is through direct 
instruction and models, and to a certain extent assessment, that students are 
instructed in the processes of collecting high quality field data. 

Assessment 

Field Data Materials and Equipment 

During the second class meeting, students are instructed to display their 
required field data collection materials and equipment. Along with their display, 
they are asked to indicate on the Field Data Materials and Equipment Assessment 
form whether they do in fact have each listed item or not. (A copy of the form is 
presented in Figure 1.) If an item is not present, then students are instructed to 
indicate at the bottom of the form when they will have the missing items. The 
completed forms are set by their displays. 
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Classe sessions are three hours long. Typically, there is a break about half way 
through each session. Before the break, students are instructed to clear their tables 
of personal items and display their materials and equipment on the tables during 
the break. It is after the break the assessment takes place. The assessment process 
takes about one-half hour for a class of about fifteen students. 

After displaying their materials and equipment, students are instructed to 
examine the displays of other students. During this time, the teacher examines each 
display and completed list. Following each examination, the teacher comments to 
individual students about appropriate, inappropriate, and/or missing items. As 
noted above, when items are missing, students are instructed to indicate on the 
form when items will be secured. Complete forms are collected. 

Field Data Organization and Quantity 

At the end of the first five weeks of the course, students are requested to 
prepared their field data for display during the following class session. Students are 
given a form entitled Field Data Assessment which lists the characteristics of 
complete and well organized field data in term of various categories. (A copy of the 
form is presented in Figure 2.) Student are also instructed to prepare their field data 
for assessment in terms of the characteristics on the form. Additionally, students 
are asked prior to the next class meeting to assess and evaluate their field data in 
terms of the characteristics listed on the form. Students are instructed to include 
their completed forms with their field data displays. Photographs of previous 
displays are shown to the class. During the sixth class session, as with the displaying 
of the field data collection materials and equipment, students are instructed to 
remove all personal items from the classroom tables and arrange their displays 
during the class break. Following the break, assessment takes place. Students are 
instructed to examine each other's displays in terms of individual and inventive 
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Figure 2: Field Data Assessment Form 

Introduction to Qualitative Research In Education Field Data Assessment 
Donald B. Reed 

Name: Date: 

(Print last, first, middle initial) 

Employing the following categories, assess your field data In terms of each 
category by circling the appropriate numerical rating. 



Categ or y 

1. Field note binder flat and fileable 

2. Field note paper substantial 

3. Notebook dividers, four 

4. Notes recorded in dark ink 

5. Extensive and detailed notes 

6. Pages of field data bound 

7. Title page included 

8. Table of contents included 

9. Prges dated 

IB. Pages numbered 

1 1. Margins tvide 

12. UJhite spaces within notes Included 

13. Diagrams included 

14. Notes filled-in with colored ink 

15. File for field documents 

16. Field documents dated 

17. Field documents titled 

18. Surveys contained In document file 

19. Photographs Included 

20. Tape recordings labeled and Indexed 

21. Research journal included 

22. interview note included and filled in 



Assessment 



Unacceptable Acceptable 



1 


2 


3 


4 


5 


1 


2 


3 


4 


5 


1 


2 


3 


4 


5 


1 


2 


3 


4 


5 


1 


2 


3 


4 


5 


1 


2 


3 


4 


5 


1 


2 


3 


4 


5 


1 


2 


3 


4 


5 


1 


2 


3 


4 


5 


1 


2 


3 


4 


5 


1 


2 


3 


4 


5 


1 


2 


3 


4 


5 


1 


2 


3 


4 


5 


1 


2 


3 


4 


5 


1 


2 


3 


4 


5 


1 


2 


3 


4 


5 


1 


2 


3 


4 


5 


1 


2 


3 


4 


5 


1 


2 


3 


4 


5 


1 


2 


3 


4 


5 


1 


2 


3 


4 


5 


1 


2 


3 


4 


5 



23. Field observation notes Included and filled In 
Calculate average assessment 



2 3 4 5 
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ways in which others have organized and managed their field data within the 
specifications contained in the assessment form. 

Assessment takes about forty-five minutes. During the assessment period, 
students at various times attend their own displays and examine the displays of the 
other students in the class. Students are interested in explaining the inventive ways 
in which they each have organized, catalogued, and filed their own field data. 
Similarly, students are interested in observing the ways other students have met the 
course standards for field data collection and presentation ii unique and 
personalized ways. It is not uncommon for animated conversations regarding 
various ways of organizing and presenting field data to occur during the field data 
display 

While the students are examining other students' displays, the teacher 
systematically examines each display. First the assessment form is examined. Then 
the filed data is examined in terms of the assessments on the form. The presence 
and absence of categories of data are examined. The volume of data within 
categories is also examined. Students are asked to explain categories in which they 
have given themselves low ratings. It is not uncommon for students to point out 
deficiencies as well as unique ways they have invented to organized and file their 
field data. In some cases, students seemingly assess their field data too harshly or 
too leniently. In these cases, the teacher suggests that changes be made on the 
assessment forms. When the assessments are completed, the teacher photographs 
the displays and members of the class. Figures 3 and 4 are photographs of field data 
displays conducted in several of the introductory qualitative research classes. The 
completed assessment forms are collected by the teacher following photographs at 
the end of the display period. 

Field Data Substance and Quality 

Assessment of the quality of individual student's field data is only in part 
conducted during the assessment period. Neither time nor the very individual way 
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Figure 3 : A Student Displaying Her Field Data 
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Figure 4: Field Data Displays in a Classroom 
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in which students write and present field notes allows for much examination of the 
substantive quality of their field data. During the assessment period, however, the 
examination of the presence or absence of various categories of data, the volume of 
data in the various categories, as well as the presence of detailed diagrams do 
indicate the substantive quality of students' field data. During the data collection 
phase of the study, it is through the class activities presented above that the quality 
of students’ individual field data is also assessed. The final report of a study as well 
as the public oral presentation of the report and distribution of the report also 
provide evidence of the quality of students' individual field data. 

A Theoretical Perspective Regarding Assessment 
Theoretical Concepts 

Classroom teaching involves both production and maintenance activities 
(Mitchell & Spady, 1977). The student production of an independent and original 
qualitative research study involves both instruction and assessment. It is important 
to examine briefly the assessment process from a theoretical perspective. 

Instruction is the process whereby a teacher intentionally attempts to 
encourage students to acquire information and/or knowledge. Instruction may 
involve presenting information and/or knowledge to students, or it may involve 
guiding students through new and different experiences so that they construct their 
own knowledge (Jackson, 1986). 

Assessment is the examination of evidence of production. Evaluation is a 
term frequently used along with assessment, as in assessment and evaluation. 
Evaluation occurs when a judgment is made during or following an assessment by 
an individual who is both required to and has the expertise to make a judgment of 
the quality of a production. Evidence is judged employing a variety of standards, 
typically organizational regulations, professional or academic norms, technical 
standards, or the personal values of the individual evaluating. Usually evaluations 
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are documented and filed, and the substance of evaluations is confidential. When 
evaluation occurs, it is assumed that an individual or individuals will formulate a 
production to conform with organizational regulations, professional norms, 
technical standards, and/ or the personal values of others. Conformation occurs 
because those who are judging the production can sanction both the production and 
those involved in the production, and/ or because the individual, group, or 
organization engaged in production wants its production to conform to certain 
internalized standards. 

The term accountability, although ue+ usually used along with the term 
assessment, can be linked with the term assessment, as in assessment and 
accountability. Here the term accountability means the provision for public or semi- 
public scrutiny of evidence of production. Scrutiny of the evidence is voluntary, not 
mandatory. Public scrutiny may or may not be limited to an organization, a 
profession, or a group. As with evaluation, evidence is judged employing a variety 
of standards; typically organizational regulations, professional or academic norms, 
technical standards, or the personal values of an individual or individuals. 
Accountability does not require documentation. Similar to evaluation, when 
accountability processes are employed, it is assumed that an individual or 
individuals will formulate a production to conform with organizational 
regulations, professional norms, technical standards, and/or the personal values of 
others. Here again, conformation occurs because those who are judging the 
production can sanction both the production and those involved in the production 
and/ or because the individual, group, or organization engaged in production wants 
production to conform to certain internalized standards. 

It is important to note that during instruction, assessment usually takes place. 
As instruction proceeds, teachers confirm or disconfirm students’ knowledge. 
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Following disconfirmation, typically new instruction occurs. Also, it is important to 
note that during assessment, instruction may take place. Through assessments, 
students may gain knowledge from their teachers or others. 

Assessment and Instruction 

Within the assessment of field data collection materials and equipment and 
the assessment of completed field data, there is a reciprocal instructional component 
as well as the more obvious assessment component. Within the instructional 
component of the assessment process, there are three distinctly different types of 
instruction. 

When students display, first, their data collection materials and, secondly, 
their complete field data, they examine their own displays. These examination are 
conducted in terms of an assessment form. While completing the assessment 
forms, students instruct themselves regarding the salient aspects of appropriate data 
collection materials and equipment and of appropriately completed and organized 
field data. Also, when students display, first, their data collection materials and, 
secondly, their complete field data, they examine each other's displays. This 
examination gives rise to conversations between and among students. Through the 
examinations and related conversations, students instruct each other regarding 
appropriate data collection materials and equipment as well as appropriate ways to 
organize and present field data. Additionally, as the teacher examines each 
student's display, the teacher provides specific instruction in terms of confirmation 
and disconfirmation. Where disconfirmation occurs, further instruction is 
provided. 

Assessment and Evaluation 

In this section of the paper and the one which follows, only the assessment of 
field data will be discussed. Because the assessment and evaluation of field data 
collection materials and equipment is very much a truncated process of the 
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assessment and evaluation of the field itself, only the assessment of field data with 
respect to evaluation and accountability will be presented. 

Within the assessment process itself, there are two different types of 
assessment which occur simultaneously. Related to each of the types of assessment 
is a type of evaluation. As students display their productions (their completed field 
data) they assess their own displays; then the teacher assesses each student's display. 
Following the assessment, students are required to evaluate their own work by 
completing an evaluation form. The teacher evaluates student production by 
examining and judging the completed field data in terms of the evaluation form. 
Each student's evaluation form and completed field data are assessed by other 
students in the class, but the students do not conduct evaluations. 

Assessment and Accountability 

It is during the semi-public display of field data, that students become 
accountable to others in the class for the character of their field data. As students 
examine each display, they make judgments about each other's field data. These 
judgments are not required but nonetheless are made. The judgments are taken 
seriously because the students regard each other as more than novices with respect 
to field data character and organization. Student displays which do not meet, or 
barely meet, standards are immediately recognized but with little or no comment 
from others. Displays which clearly meet the standards are noticed. However, it is 
the students who develop, within standards, interesting, unique, and useful ways to 
organize and present their field data who are recognized and praised by other 
students. 

Quality and Substance of Field Data 

Although the purpose of this paper is not to discuss assessment, evaluation, 
and accountability with respect to the quality and substance of field data, some 
discussion of the substance and quality of field data with respect to assessment, 
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evaluation, and accountability can be presented. As noted above, the assessment of 
field data in terms of quality and substance is very difficult because of the length of 
time such assessments take and because of the very idiosyncratic way in which field 
data are recorded. The teacher does examine and assess samples of field data for 
quality during the data display period and offer tentative evaluations. However, it 
is through the four drafts of a report of the study and the final semi-public oral 
presentation of the report that assessment and evaluation of the quality of the field 
data for individual students are made. It is also through the semi-public 
presentations and the availability of all the individual reports of the qualitative 
studies that students feel held accountable for the quality of their field data. 

Conclusion 

Class displays of field data for individual qualitative research studies provide 
a way for both teacher and individual students to assess and evaluate their field 
data. Perhaps more importantly, class displays of student field data provide a means 
whereby individual accountability for field data with respect to peers in terms of 
both technical and substantive qualities can take place. It is accountability which 
raises student production of quantity, quality, and organization of field data to a 
higher level than otherwise would be the case. 
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